This PR adds pytest --collect-only to the CI for Python 3 and none of the changes should make a difference in Python 2. More changes will be needed before the tests work but this will at least ensure that it's possible to import most of DIRAC. All of the changes should be transparent in Python 2 and future PRs can then start enabling parts of the unit tests by adding -k mytests to the command.

I've constructed it to be easy to review as individual commits instead of in one go. I've split them into two groups:

Simple renaming of modules that is handled using six

• 62b10d7: Replace "import thread" with "from six.moves import _thread as thread"
• b9b6299: Replace "import urlparse" with "import six.moves.urllib.parse"
• 4bdc9b0: Replace "import Queue" with "from six.moves import queue as Queue"
• 331c7ed: Replace "itertools.izip_longest" with "six.moves.zip_longest"
• 58d240d: Get urllib2.URLError from six.moves.urllib_error
• 7ece6c3: Replace StringIO with six.StringIO and six.BytesIO
  • In Python 2 six.StringIO and six.BytesIO are both StringIO.StringIO. In Python 3 they are io.StringIO and io.BytesIO but it's not practical to use these in Python 2 as StringIO requires unicode objects as input which isn't worth it. I've tried to guess at which one will be appropriate for Python 3 but more changes will definitely be needed here to make the tests pass once they're being executed.

Other changes

• 8a4eaf1: Fallback to subprocess.getstatusoutput for commands.getstatusoutput
  • These should be rewritten to use subprocess32 properly rather than using the long-deprecated getstatusoutput. (the subprocess32 doesn't include Python 3's subprocess.getstatusoutput)
• d285b56: Encode strings as UTF-8 before using hashlib.md5
• 604de9c: Allow importing DEncode in Python 3 without actually porting it
• 0716316: Minimal types fixes to make pytest discovery functional
• 69f058b: Fix string type checking in Core/Utilities/Time.py
• 846d8a6: Escape backslash in Core/Utilities/Grid.py
• 4f3f839: Remove Python version check on import
  • DIRAC errors on import for anything except Python 2.7 but I don't see any reason to keep this check given dirac-install will force people to get Python 2.7 from DIRACOS.
• 5a66509: Remove MySQLdb.server_init if using mysqlclient
  • MySQL-python was abandoned a long time ago and never supported Python 3. It looks like mysqlclient is almost a drop in replacement so that's what I've gone with for now. The only change it seems to need is this commit.
• c8c1f9d: Run pytest discovery with Python 3
  • I add a new environment-py3.yml file to manage the Python 3 dependencies. It's ugly to list them in yet another place but I think it's necessary given things won't be exactly the same.
  • fts-rest Doesn't seem to support Python 3 at all. I've made enough changes in a fork to make it possible import and that's what I use here. Porting it properly to using six should only take a few hours but I didn't find any tests and we should ask the developers to see what their plans are.

BEGINRELEASENOTES

*Python 3
NEW: Run pytest test collection for Python 3

ENDRELEASENOTES